Method for evaluating and correcting the image data of a camera system

ABSTRACT

The image data of a camera system is corrected. The camera system includes a color camera which supplies three output signals to three separate color channels, in the form of output signal vectors. The output signal vectors, the coefficients of which represent the output signals supplied by the color camera to the three color channels in a specific position lying in the range of observation, are multiplied by a correction matrix, particularly a square one. The corrected output signal vectors are then processed further in the camera system.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is the U.S. National Phase under 35 USC 371 of PCT/DE03/00570, filed Feb. 22, 2003; published as WO 03/073748 A2 on Sep. 4,2003 and claiming priority to DE 102 08 285.5 filed Feb. 26, 2002, thedisclosures of which are expressly incorporated herein by reference.

FIELD OF THE INVENTION

The present invention is directed to a method for evaluating andcorrecting the image data of a color camera system that is particularlywell suited for inspecting color printed products.

BACKGROUND OF THE INVENTION

Camera systems are widely used for various applications, particularly inthe printing industry. For example, camera systems can be used forregistration measuring systems, inspection systems or web monitoringsystems. In many cases digital image sensors such as CCD cameras areemployed as color cameras, whose light-sensitive pixels provide threeoutput signals corresponding to the colors recorded in the observationarea via three separate color channels, usually for the colors red,green and blue.

In connection with the known camera systems, a problem occurring duringthe inspection of colored printed products is that the color dataprovided by the color cameras often do not correspond to the color senseof the human eye. Unprocessed image data from these color cameras areunsatisfactory in the areas of color balance, brightness, contrast andreproduction of the color hue when compared to human color perception.Both optical devices and illumination devices have shortcomings and thespectral sensitivity distribution of CCD cameras exacerbates the colorbalance problems, brightness problems, contrast problems and color hueproblems. The sensitivity distribution of the cameras used does notalways agree with the sensitivity distribution of the human eye, and sothe image data provided by the color camera makes a false visualimpression during later processing, such as when viewed on a videodisplay monitor.

A method for correcting image data by means of a correction matrix isdisclosed in U.S. Pat. No. 5,189,511.

U.S. Pat. No. 4,930,009, U.S. Pat. No. 5,331,441 and U.S. Pat. No.6,278,533 also disclose methods for correcting image data of a camerasystem with a color camera, using a correction matrix.

SUMMARY OF THE INVENTION

The object of the present invention is evaluating and correcting theimage data of a camera system.

In accordance with the invention, this object is attained by a methodfor correcting the image data of the camera system, particularly suitedfor the inspection of colored printed products having a color cameraproviding three output signals corresponding to the colors recorded, forthree separate color channels in the form of output signal vectors. Theoutput signal vectors, whose coefficients (R, G, B) represent the outputsignals provided by the color camera for the three color channels at adefined position in the observation area, are multiplied with acorrection matrix, preferably a quadratic correction matrix.

A prerequisite for processing the image data of the camera system inaccordance with sensitivity is that as many as possible of the colorsrecorded by the color camera are arranged sufficiently correctly inregard to hue, saturation and brightness in a color range whichcorresponds to the color sensitivity of the human eye. CommissionInternationale de l'Éclairage (CIE), is an international clearinghousefor color research at universities and research laboratories. A recentmilestone in this effort was the CIE L*a*b* color system (CIELAB forshort), first published in 1976. In this method, the so-called CIELABcolor range is particularly suitable and is widely used in the printingindustry. A measurement number for the accuracy of color differences inaccordance with sensed color differences is provided in the CIELAB colorrange by determining the geometric distance between the nominal and theactual value of CIELAB color model variables L, A and B (Delta E).

In the method of the present invention the output signal vectors aremultiplied by, preferably, a quadratic correction matrix, and so theimage data can be changed in a simple way so that they approach thecolor perception of the human eye. Multiplication by a correction matrixprovides a relatively accurate arrangement of all printing colors in abasically arbitrary color range. Moreover, the conversion bymultiplication with a correction matrix can be performed on camera imagedata in such a simple way that practical and cost effectiveimplementation is possible even for large amounts of image data.

The coefficients of the correction matrix determine the quality of thecorrection since the output signal vectors are transformed in differentways, depending on the selection of these coefficients. For example, thecoefficients of the correction matrix can be determined from empiricalvalues and can then be permanently stored in a computer memory. In orderto be able to match the coefficients of the correction matrix variablyto different marginal values, for example when compensating for a knowncamera, given illumination conditions or the given optical device used,an iterative approximation algorithm is used. For performing thisapproximation algorithm it is necessary to first provide a referencecolor table, for example an IT8-chart with 288 color fields. Differentreference colors are represented in the color fields. The classificationof the different reference colors in a suitable color range, for examplethe CIELAB color range, is known. By means of known transformations itis possible to calculate corresponding nominal output signals for thethree color channels from these predetermined CIELAB values for thevarious reference colors of the reference color table. As a result, areference color table is predetermined as the input value for theapproximation algorithm, and for every reference color a nominal vectorfor the three color channels is the desired result of the conversion. Inthe course of performing the approximation algorithm for determining thecoefficients of the correction matrix, the reference color table is nowrecorded with the color camera, and an output signal vector of the colorcamera is determined for each color field. The difference between theseoutput signal vectors of the color camera and the predetermined nominalvectors corresponds to the difference between the color perception ofthe human eye and the sensitivity distribution of the color camera.

Advantageously, this method includes calculation of the color correctionvalues for different illumination sources and changes among differentillumination sources. At present, the standard light source, known as aD50 is used in printing technology. By predetermining the illuminationcharacteristics of a D50 light source it is possible to match the Rec.709 color standard by conversion to the D50 standard light, so that theintensities of the non-linear R′, G′, B′ values act as if the object tobe investigated were illuminated by a D50 standard light's illumination.A measurement method interactively matches the values of the R′, G′, B′color range with the CIELAB color range, to adapt the color ranges toeach other without a real standard illumination being required. Thismethod has the advantage that in case of a change of the standard lightconditions to be expected, the change in illumination can be compensatedfor immediately by executing the method of the present invention incomputer software.

The starting point for an iteration is a correction matrix whosecoefficients are preset as initial values. These initial values caneither be selected completely randomly or can be set to pre-definedempirical values. In the first iteration step, the correction matrix ismultiplied by all output signal vectors, and the corrected output signalvectors obtained are placed in the computer's buffer storage. Next, ifthe corrected output signal vectors approach the preset nominal vectors,the coefficients of the correction matrix are slightly changed, and themultiplication is performed again. The change of the coefficients of thecorrection matrix is here accepted only if the corrected output signalvectors approach the preset nominal vectors.

Next, the approach of the corrected output signal values is compared tothe preset nominal vectors for each iteration step in order to be ableto decide, on the basis of this comparison, whether the changes of thecoefficients of the correction matrix made in this iteration step are tobe used or discarded. In the comparison or assessment method of thepresent invention, the difference value between the corrected outputsignal value and the nominal vector predetermined for the color field isdetermined for each color field of the reference color table, and thesum of all these difference values is added together. The change of thecorrection coefficient of the correction matrix in the last iterationstep is then used only if the sum of all difference values in the lastiteration step has become smaller. But if the sum of all differencevalues by changing the coefficient of the correction matrix has becomelarger in the last iteration step, the change in the coefficients is notused and discarded. By checking the sum of the difference values overall reference colors, it is easily possible that the difference forindividual reference colors increases in the course of an iterationstep. However, as a whole, minimizing the difference values over allcolor channels is dependably assured.

Another problem with existing camera systems is the correct setting ofthe color balance, i.e. the correct weighting of the three colorchannels in respect to each other. To be able to set the color balanceof the individual color channels in relation to each other, a correctionvector can be added to each output signal vector, and at the same timethe coefficients of each output signal vector can be multiplied by threecolor channel-dependent correction factors. This correction of the threecolor channels corresponds to a linear representation of the individualcoefficients of the output signal vectors.

A particularly good color balance is achieved if the correction vectorand the three color channel-dependent correction factors are selected sothat the corrected output signal vectors correspond to a standard. Thecorrected output signal vectors are obtained by applying the correctionwith the correction vector. The three correction factors for thereference values black and white preferably correspond exactly to thenominal vectors preselected for these two color fields. This means, inother words, that the linear representation of the output signal vectorsis selected in such a way that corrected results are obtained for thetwo reference grey-scale values, black and white, which correspond tothe contrast perception of the human eye. This linear representation isapplied to all output signal vectors, so that brightness and contrastare automatically corrected in the entire color spectrum.

CCD color cameras with a plurality of pixels, which are arranged flat orin line shapes, are particularly suitable for executing the method inaccordance with the invention. These CCD cameras also have three colorchannels and provide output signal vectors by pixels as the image data,whose coefficients represent each of the three output signals for thethree color channels (red, green, blue). If, for example, the CCD colorcamera is provided with a million pixels, this corresponds to an imagedata amount of one million output signal vectors, each with threecoefficients, for each image.

When using CCD color cameras, color distortion and a drop of intensitycan occur, in particular at the edges of the camera images. Thesedistortions are created by the lenses used. It is possible to use aso-called shading correction for correcting this drop in intensity. Tothis end, three color channel-dependent correction factors are presetfor each pixel. By multiplying these pixel-dependent correction factorswith the coefficients of the output signal vectors, it is possible tocompensate the pixel- specific color distortions, or a drop in intensitybased on the structural type, in the different areas of the CCD chips.

These pixel-specific, color channel-dependent correction factors can forexample be experimentally determined in a simple way, in that theobservation area of the CCD camera is covered in a homogeneous material,in particular a homogeneous white material, and an output signal vectoris determined for each pixel by triggering the camera. The output signalvector having the highest coefficients, and therefore represents thebrightest point in the observation area, is then filtered out of allthese output signal vectors. But since the observation area is coveredby a homogeneous colored material, all pixels should provide outputsignal vectors which essentially agree identically with each other.Therefore the respective differences are based on color distortions oron a drop in intensity because of the structural type. To compensate forthis, correction factors are now selected for each color channel of eachindividual pixel, which assure that during the recording of thehomogeneous colored material all output signal vectors correspond to theoutput signal vector at the brightest spot in the observation area.

Color distortions in particular greatly depend on the illuminationconditions in the observation area. For excluding error sources becauseof a change of the illumination conditions, the illumination during theexperimental determination of the pixel-specific color channel-dependentcorrection factors should correspond to the illumination during thesubsequent use of the camera system.

In many applications of the method of the invention, the correctedoutput signal vectors obtained by correcting the output signal vectorsof the color camera are used for controlling the three separate colorchannels of a color image monitor. The representation of the colors on acolor image monitor also poses the problem that the representationalcharacteristics of most color image monitors do not correspond to thecolor perceptions of the human eye. This is based in particular on thefact that the brightness conditions of monitors as a rule are notlinear, i.e. the intensity of the light reproduced on the screen is anon-linear function of the electric input signals. In other words thismeans that undesired distortions in the color image on the displayscreen occur in case the output signal vectors which, in accordance withthe invention, have been corrected according to the color perceptions,are simply transmitted to the color image monitor and are displayedthere without taking the non-linearity of the brightness conditions intoconsideration.

To prevent such color distortions in the display on a color imagemonitor, the coefficients of the corrected output signal vector as basiscan be exponentiated by a factor γ. By means of this non-linearconversion of the coefficient of the corrected output signal vectors itifs possible to compensate the non-linearity of the brightnessconditions of most color image monitors. For most color image monitors,the factor γ must be selected in the value range between 0.3 and 0.5 andpreferably, γ is selected to be approximately 0.45.

In order not to have to calibrate the illumination source to a standardlight source when using corresponding camera systems, it is possible inaccordance with invention to perform a further correction step. In thiscorrection step the coefficients of the output signal vectors areconverted in such a way that the result corresponds to those outputsignal vectors, which would be obtained when illuminating theobservation area with a standard light.

An exemplary embodiment of the invention is represented in the drawingsand will be described in greater detail in what follows.

BRIEF DESCRIPTION OF THE DRAWING

FIG. 1, the sole drawing figure, shows the different method steps in thecourse of executing an embodiment of the method, in accordance with theinvention.

DESCRIPTION OF THE PREFERRED EMBODIMENT

Referring to FIG. 1, a printed product 03, is arranged in an observationarea 02 and is imprinted in colors. An image of printed product 03 isrecorded with a color camera 01, preferably a CCD camera. A CCD chip isprovided in the color camera 01, which converts the image information inthe observation area 02 into electronic image data. In the course ofthis conversion, an output signal vector 04 is generated by eachlight-sensitive pixel of the CCD chip. A number of output signal vectors04 corresponding to the number of pixels on the CCD chip are madeavailable by the color camera 01 for further processing.

Each output signal vector 04 preferably includes three coefficients R, Gand B. The coefficients R, G and B correspond to the color values of thethree color channels red, green and blue, wherein the color of theprinted product 03 at the position in the observation area which wasrecorded by the corresponding pixel, corresponds to the mixture of thethree color channels red, green and blue.

The output signal vectors 04 have an index corresponding to thearrangement of the respective pixel on the CCD chip, are furtherprocessed in the form of raw data in a first correction module 06 formatching the color balance, brightness and contrast. For this purpose,the coefficients R, G, B of the output signal vector 04 are allmultiplied by the color channel-dependent correction factors K₁, K₂ andK₃, and a correction factor 07 with fixed value coefficients a₁, a₂ anda₃ is added to the resultant vector. The corrected output signal vectors04 a are created by means of this calculation operation, which improvesthe color balance, the brightness and the contrast of the image data.This aim is achieved because the color channel-dependent correctionfactors K₁, K₂ and K₃, as well as the coefficients a₁, a₂ and a₃ of thecorrection vector 07, are selected to meet previously selectedstandards. In the exemplary embodiment, when recording the CIELABreference or grey scale values for black and white, the output signalvectors 04 generated in the process by the color camera 01 aretransformed such that the corrected output signal vectors 04 a willcorrespond to nominal values such as those resulting from the conversionof the known CIELAB color values to nominal vectors.

In the next step, the corrected output signal vectors 04 a are passed onto a second correction module 08. In the correction module 08 eachoutput signal vector 04 a is multiplied with, preferably, a 3×3correction matrix 09, and the corrected output signal vectors 04 b arecalculated from this. In the preferred embodiment, the coefficients K₄to K₁₂ of the correction matrix 09 are determined in advance using asuitable iteration process so that the image information contained inthe output signal vectors 04 a approximates the color perception of thehuman eye.

In the next step, the corrected output signal vectors 04 b are passed onto a third correction module 10. In the third correction module 10,color channel-dependent correction factors for each pixel are stored ina computer memory or data bank, and are multiplied with the coefficientsR, G and B for matching the intensity values, which are a function ofthe position of the respective pixels. As a result, the corrected outputsignal vector 04 b of the first pixel is multiplied by the correctionfactors K₁₃, K₁₄ and K₁₅ in order to calculate therefrom a correctedoutput signal vector 04 c for the first pixel. The calculation of thecorrected output signal vector 04 b takes place pixel by pixel.Therefore the number of the pixel-specific correction factorscorresponds to three times the number of the pixels.

In the next step, the corrected output signal vectors 04 c are passed onto a fourth correction module 11, for video monitor non-linearcharacteristic compensation. In the fourth correction module 11 thecoefficients R, G, B of the corrected output signal vectors 04 c areraised to the power of a factor γ or exponentiated by a factor γ, andthe corrected output signal vectors 04 d are calculated from the result.The non-linear brightness transmission function of video monitor 12 istaken into consideration by performing this exponentiation step.

As a result, correcting the output signal vectors 04 in the correctionmodules 06, 08, 10 and 11 makes the color images represented on thescreen of the color image video monitor 12 match the color perception ofthe human eye so that the visual impression, when viewing the display atthe color image monitor 12, corresponds closely to the color perceptionwhich would arise when directly viewing the printed product 03.

While preferred embodiments of a method for evaluating and correctingimage data, in accordance with the present invention, have been setforth fully and completely hereinabove, it will be apparent to one ofskill in the art that various changes in, for example, the specificcorrected output signal vector, the type of printed product observed,and the like can be made without departing from the true spirit andscope of the present invention which is accordingly to be limited onlyby the appended claims.

1. A method for correcting the image data of a camera system for theinspection of colored printed products, comprising the steps of: (a)providing an observation area adapted to receive and illuminate aprinted product; (b) providing a color camera adapted to generate first,second and third output signals for first, second and third separatecolor channels in the form of output signal vectors, said first, secondand third output signals corresponding to image colors recorded; (c)aiming said color camera at said printed product in said observationarea; (d) generating said first, second and third camera output signalscorresponding to a recorded printed product image; (e) multiplying saidfirst camera output signal by a first color-channel dependant correctionfactor and adding the result to a first correction vector to generate afirst camera corrected output signal vector; (f) multiplying said secondcamera output signal by a second color-channel dependant correctionfactor and adding the result to a second correction vector to generate asecond camera corrected output signal vector; (g) multiplying said thirdcamera output signal by a third color-channel dependant correctionfactor and adding the result to a third correction vector to generate athird camera corrected output signal vector; (h) defining a cameracorrected output signal vector matrix comprising said first, second andthird camera corrected output signal vectors; (i) defining a correctionmatrix comprising selected correction coefficients; said selectedcorrection coefficients being selected by an iterative approximationprocess and compared to a reference color table; and (j) multiplyingsaid camera corrected output signal vector matrix with said correctionmatrix to generate a second corrected output signal vector.
 2. Themethod for correcting the image data of claim 1, wherein said correctionmatrix comprises a quadratic correction matrix.
 3. The method forcorrecting the image data of claim 2, wherein said correction matrixcomprises a 3×3 quadratic correction matrix having the selectedcorrection coefficients; said selected correction coefficients beingselected by the iterative approximation process and compared to thereference color table of data representing an approximation of the colorperception of a human eye.
 4. The method for correcting the image dataof claim 1, wherein said color camera's first, second and third outputsignals are for red, green and blue separate color channels in the formof the output signal vectors, said first, second and third outputsignals corresponding to red, green and blue colors recorded.
 5. Themethod for correcting the image data of claim 4, wherein said cameracorrected output signal vector matrix comprises red, green and bluecamera corrected output signal vectors.
 6. The method for correcting theimage data of claim 1, wherein said first, second and thirdcolor-channel dependant correction factors are selected to match colorbalance among red, green and blue colors.
 7. The method for correctingthe image data of claim 1, wherein said first, second and thirdcolor-channel dependant correction factors are selected to correctcontrast and brightness in said first, second and third camera correctedoutput signal vectors.
 8. The method for correcting the image data ofclaim 1, further comprising: (k) assessing said second corrected outputsignal vector by placing a reference color table in said observationarea and recording first, second and third reference color outputsignals and comparing said second corrected output signal vector withthe reference color table data.
 9. The method for correcting the imagedata of claim 8, further comprising: (l) selecting, in an iterativeprocess, new values for said correction matrix's selected correctioncoefficients.
 10. The method for correcting the image data of claim 9,further comprising: (m) again assessing said second corrected outputsignal vector when recording said first, second and third referencecolor output signals and again comparing said second corrected outputsignal vector with the reference color table data, to determine whethersaid new values for said selected correction coefficients have morenearly approximated said reference color table data with the newestiteration of said second corrected output signal vector.
 11. The methodfor correcting the image data of claim 10, further comprising: (n)keeping said new values for said selected correction coefficients storedin a computer memory only if said new values for said selectedcorrection coefficients have more nearly approximated said referencecolor table data with the newest iteration of said second correctedoutput signal vector.
 12. The method for correcting the image data ofclaim 1, further comprising: (k) defining a pixel-by-pixel correctionmatrix comprising selected pixel correction coefficients; said theselected pixel correction coefficients being selected to match intensityvalues.
 13. The method for correcting the image data of claim 12,further comprising: (l) multiplying said camera corrected output signalvector matrix with said pixel-by-pixel correction matrix to generate athird corrected output signal vector.
 14. The method for correcting theimage data of claim 12, further comprising: (m) defining a displaycompensating exponent for a selected image or video display's non-linearcharacteristics.
 15. The method for correcting the image data of claim14, further comprising: (n) generating a fourth corrected output signalvector by raising said third corrected output signal vector to the powerof said display compensating exponent.
 16. The method for correcting theimage data of claim 14, further comprising: (o) displaying a fourthcorrected output signal vector on said selected image or video display.17. The method for correcting the image data of claim 1, furthercomprising: (k) selecting illumination source specific values for saidcorrection matrix's selected correction coefficients to compensate forthe characteristics of a selected illumination source illuminating saidobservation area, and assessing said second corrected output signalvector by placing the reference color table in said observation area andrecording first, second and third reference color output signals andcomparing said second corrected output signal vector with the referencecolor table data.
 18. The method for correcting the image data of claim17, further comprising: (l) selecting, in an iterative process, newvalues for said correction matrix's selected correction coefficients.19. The method for correcting the image data of claim 18, furthercomprising: (m) again assessing said second corrected output signalvector when recording said first, second and third reference coloroutput signals and again comparing said second corrected output signalvector with the reference color table data, to determine whether saidnew values for said selected correction coefficients have more nearlyapproximated said reference color table data with the newest iterationof said second corrected output signal vector; and (n) keeping said newvalues for said selected correction coefficients stored in a computermemory only if said new values for said selected correction coefficientshave more nearly approximated said reference color table data with thenewest iteration of said second corrected output signal vector.
 20. Themethod for correcting the image data of claim 1, further comprising: (k)selecting optical component specific values for said correction matrix'sselected correction coefficients to compensate for the characteristicsof a selected optical component used in recording said image, andassessing said second corrected output signal vector by placing a CIELABreference color table in said observation area and recording first,second and third reference color output signals and comparing saidsecond corrected output signal vector with CIELAB reference color tabledata; (l) selecting, in an iterative process, new values for saidcorrection matrix's selected correction coefficients; (m) againassessing said second corrected output signal vector when recording saidfirst, second and third reference color output signals and againcomparing said second corrected output signal vector with CIELABreference color table data, to determine whether said new values forsaid selected correction coefficients have more nearly approximated saidCIELAB reference color table data with the newest iteration of saidsecond corrected output signal vector; and (n) keeping said new valuesfor said selected correction coefficients stored in a computer memoryonly if said new values for said selected correction coefficients havemore nearly approximated said CIELAB reference color table data with thenewest iteration of said second corrected output signal vector.